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REMARKS 



Applicant has amended claims 1, 5, 7, 8, 11, 12, 15, 20, 22, 23, 26, 27 to more 
particularly point out and distinctly claim the subject matter of the invention. Applicant has also 
added two new claims 28 and 29. 

Applicant has submitted a new set of formal drawings, replacing FIGs 1-7, and amended 
the specification to conform with these replacement drawings. No new matter has been added. 
Attached is a marked-up version of the changes being made by the current amendment. 

Applicant submits that all of the claims are now in condition for examination, which 
action is requested. Enclosed is a $36.00 check for excess claim fees. Please apply any other 
charges or credits to Deposit Account No. 06-1050. 



Fish & Richardson P.C. 
500 Arguello Street, Suite 500 
Redwood City, California 94063 
Telephone: (650) 839-5070 
Facsimile: (650) 839-5071 



Respectfully submitted. 




Fredrik MoUbom 
Reg. No. 48,587 
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Versioim with marlldiags to show chaiBges made 



In the specification: 

Paragraph beginning at page 5, line 25 has been amended as follows: 
As shown in FIG. 2A, the network server 42 transmits the web page 70 to a user on client 
computer 14, 16, 18 to allow the user to send images 1 1 to the server 12. The user may add 
images to the web page by clicking on an add button 72. The web page 70 includes images 1 la- 
lid that have been added to the web page 70 using the add button 72. The images lla-1 Id 
depict overlapping segments of a view of a [lake] U;ee. The user may transmit each of the images 
lla-1 Id by clicking on an upload button 74a-74d that corresponds to the image. The user directs 
the server 12 to create a panoramic image from the uploaded images by clicking on a create 
button 76, causing client computer 14, 16, 18 to transmit the images to the server 12 through the 
network 13. The input interface 44 of the network server 42 receives the images 1 la-1 Id 
through the network interface 30 and conveys them to the image input interface 49, which in turn 
conveys the images lla-1 Id to the image stitching software 48. The images stitching software 
48 merges the images 1 la-1 Id to form a panoramic image of the entire view of the scene, which 
it conveys to the output interface 46 of the network server 42. 

Paragraph beginning at page 6 , line 22 has been amended as follows: 
For example, as shown in FIGS. 4A and 4B, the positioning module 50 uses the two 
image positioner 60 to determine how much a first image 80a needs to be moved relative to a 
second image 80b so that a certain object depicted in both of the images 80a, 80b has its 
depiction in the second image 80[a]b on top of its depiction in the first image 80[b]a. In FIG. 
4A, the image 80b must be moved 68 pixels to the right and 2 pixels upwards so that a branch 82 
which is depicted in both image 80a, 80b has its depiction in the second image 80b on top of its 
depiction in the first image 80a. This ensures that the two images 80a, 80b are positioned so that 
the images 80a, 80b continue into each other as seamlessly as possible without altering the pixels 
of the images. 
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Paragraph beginning at page 9, line 1 1 has been amended as follows: 
The dividing-line determiner 54 (FIG. 1) determines (212) an outline 85 (FIG. 4F) of a 
composite image formed by aligning the current image 80b and the reference image 80a (as 
previously described with reference to FIG. 4A). The dividing-line determiner 54 also 
determines a pair of points 87a, 87b where the outlines of the aligned images intersect, thereby 
defining (214) a line 89 that joins the points 87a, 87b and divides (216) the panoramic outline 85 
into two sections 81, 83 (216). If the outlines of the aligned images intersect at more than two 
points, the dividing-line determiner 54 selects the two intersection points that are furthest apart 
from each other to define the dividing line 89. The dividing-line determiner 54 then determines 
(218) which one of the two sections 81, 83 has less of the current image 80b that is not 
overlapped by the reference image 80a and sets (220) that section [87a] 84 of the current image 
80b to be invisible. In the example of FIG. 4F, the section 83 has none of the current image that 
is not overlapped by the first image 80a. Consequently, the portions of the image profile 85 
contained within the section 84 are set invisible, leaving the hashed section [82] 81 of the image 
80b visible. 

Paragraph beginning at page 9, line 25 has been amended as follows: 
The stitching software 48 checks (222) whether there are any more images between the 
reference image 80a and the current image 80b. If there are more images, the stitching software 
48 sets (224) the reference image to be the next image after the current reference image and 
repeats the process of setting a section of the current image 80b invisible (208-220) as described 
above. Otherwise, if there are no more images, the blending mask determiner 56 (FIG. 1) 
determines (226) the pixels within the current image that will mask out pixels of earlier images. 
Only visible pixels [81] of the current image 80b mask out pixels of earlier images 80a. 
Consequently, the mask value of pixels contained within the region 81 is set to "1", while the 
mask property of pixels contained within the region 84 is set to "0". 

Paragraph beginning at page 10, line 9 has been amended as follows: 
If there are no more images after the current image, the image blender 58 overlaps (230) 
the images 80a-80f based on the masking value to create the panoramic image 94 (FIG. 4E). 
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The section [87a] 81 of the second image 80b with a mask value of 1 is first composited on the 
first image, thereby obstructing the part of the first image that is to the right of the dividing line 
89. The portions of the third image 80c with a mask value of 90 are then composited on the 
composite image fi:om the first 80a and second 80b image to create another image, and so on, 
until the composite image 94 is created. Thus, image stitching software merges images 80a-80f 
depicting sections of a scene to create a panoramic image of the whole scene. 
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In the claims: 

Claims 1, 5, 7, 8, 1 1, 12, 15, 20, 22, 23, 26, 27 have been amended as follows: 
1 . (Amended) A method of merging images of segments of a view, comprising: 
receiving a first image representing a first segment of the view and a second image 

representing a second segment of the view, the images being received fi-om a remote location 

over a network; 

determining the position of the second segment of the view relative to the first segment of 
the view without the aid of positioning information provided by a human operator ; 

blending the first image with the second image based solely on the content of the images 
and the determined position of the second segment relative to the first segment to merge the first 
image and the second image into a panoramic image of the view; and 

transmitting the panoramic image over the network[; 

wherein the step of determining the position and the step of blending are performed 
without positioning information from a human operator]. 

5. (Amended) A method of merging a set of images , each image representing a 
corresponding [of segments] segment of a view[, comprising: 

retrieving a set of images representing a view] , the set including a first image 
representing a first segment of the view, a second image representing a second segment of the 
view, and a third image representing a third segment of the vie w, where the third segment of the 
view overlaps both the first segment and the second segment of the view, the method 
comprising ; 

[determining a first position of the second segment of the view relative to the first 
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segment of the view;] 

determining a [second] first relative position of the third segment relative to the first 
segment of the viev^ by processing the content of the third image and the first image; [and a 
third] 

determining a first overlap area of the first image and the third image based on the 
determined first relative position; 

determining a second relative position of the third segment relative to the second segment 
of the view by processing the content of the third image and the second image ; 

determining a second overlap area of the second image and the third image based on the 
determined second relative position; and 

[checking whether the first image overlaps the third image more than the second image 
overlaps the third image, based on the second position and the third position;] 

if the first [image overlaps the third image more than the second image overlaps the third 
image] overlap area is greater than the second overlap area , [blending the] offsetting the position 
of the third image relative to the first image and the second image [set of images] based on the 
determined first relative position [and the second position]; 

otherwise, offsetting the position of the third image relative to the first image and the 
second image based on the determined second relative position , 

7. (Amended) The method of claim 5 further comprising: 
[prior to blending the set of images:] 

determining which of the images is a central one and which are peripheral [ones 
of the set of] images [based on the first and second positions]; and 

[correcting perspective distortion in a first peripheral one of the images relative 
to] using the central [one of the images] image as an initial reference image in correcting 
perspective distortion in peripheral images . 

8. (Amended) The method of claim 7 further comprising: 

[prior to correcting perspective distortion in the first peripheral one of the images:] 
determining what pair-wise overlap areas exist between the central [one] image and each 
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of the peripheral [ones of the set of] images[, based on the first and second positions]; and 

selecting [the first peripheral one of the images] as the first peripheral image to have 
perspective distortion corrected a peripheral image [to maximize the] having a maximiom pair- 
wise overlap area [between] with the central image [and the first] relative to the other peripheral 
[one of the] images. 

11. (Amended) The method of claim 5, [wherein] further comprising blending the [set of 
images] third image with the first and second image, wherein the blending [further] includes: 

dividing the [second] third image into a first portion and a second portion, based on the 
first relative position; and 

compositing the first portion of the [second] third image on the first image at the first 
position to produce a composite image[;]^ the compositing causing the first portion to mask out a 
part of the first image[; 

blending the third image with the composite image to form the panoramic image]. 

12. (Amended) The method of claim 1 1 wherein blending the third image with the 
[composite image] first and second image further includes: 

dividing the [third] second image into a third portion and a second portion, based on [the 
second] a relative position of the second segment of the view relative to the first segment of the 
view; 

dividing the third portion into a fifth portion and a sixth portion, based on the [third] 
second relative position; and 

compositing the fifth portion of the third image on the composite image based on the 
second relative position to form the panoramic image, the compositing of the fifth portion 
causing the fifth portion to mask out a part of the composite image. 

15. (Amended) An article comprising a computer-readable medium on which are 
tangibly stored computer-executable instructions for merging images of segments of a view, the 
stored instructions being operable to cause a computer to: 

receive a first image representing a first segment of the view and a second image 
representing a second segment of the vie w, the images being received from a remote location 
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over a network; 

determine the position of the second segment of the view relative to the first segment of 
the view without the aid of positioning information provided by a human operator ; 

blend the first image with the second image based solely on the content of the images and 
the determined position of the second segment relative to the first segment to merge the first 
image and the second image into a panoramic image of the view; and 

transmit the panoramic image over the network. 

20. (Amended) An article comprising a computer-readable medium which stores 
computer-executable instructions for merging a set of images , each image representing a 
corresponding [of segments] segment of a view[, the instructions being operable to cause a 
computer to: 

retrieve a set of images representing a view] , the set including a first image representing a 
first segment of the view, a second image representing a second segment of the view, and a third 
image representing a third segment of the viewf;] , where the third segment of the view overlaps 
both the first segment and the second segment of the view, the instructions being operable to 
cause a computer to: 

[determine a first position of the second segment of the view relative to the first segment 
of the view;] 

determine a [second] first relative position of the third segment relative to the first 
segment of the view by processing the content of the third image and the first image; [and a 
third] 

determine a first overlap area of the first image and the third image based on the 
determined first relative position; 

determine a second relative position of the third segment relative to the second segment 
of the view by processing the content of third image and the second image ; 

determine a second overlap area of the second image and third image based on the 
determined second relative position; and 

[check whether the first image overlaps the third image more than the second image 
overlaps the third image, based on the second position and the third position;] 
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if the first [image overlaps the third image more than the second image overlaps the third 
image] overlap area is greater than the second overlap area , [blend] offset the [set of images] 
position of the third image relative to the first image and the second image based on the 
determined first relative position [and the second position]; 

otherwise, offset the third image relative to the first image and the second image based on 
the determined second relative position . 

22. (Amended) The article of claim 21 w^herein the stored instructions fiirther comprise 
instructions operable to cause the computer to: 

[prior to blending the set of images:] 

determine which of the images is a central one and which are peripheral [ones of 
the set of] images [based on the first and second positions]; and 

[correcting perspective distortion in a first peripheral one of the images relative 
to] use the central [one of the images] image as an initial reference image in correcting 
perspective distortion in peripheral images . 

23. (Amended) The article of claim 21 wherein the stored instructions further comprise 
instructions operable to cause the computer to: 

[prior to correcting perspective distortion in the first peripheral one of the images:] 

[determining] determine what pair-wise overlap areas exist between the central 
[one] image and each of the peripheral [ones of the set of] images[, based on the first and second 
positions]; and 

[selecting] select [the first peripheral one of the images] as the first peripheral 
image to be corrected for perspective distortion a peripheral image [to maximize the] having a 
maximum pair- wise overlap area [between] with the central image [and the first] relative to the 
other peripheral [one of the] images. 

26. (Amended) The article of claim 21 wherein the stored instructions further comprise 
instructions operable to cause the computer to blend[ing] the [set of images] third image with the 
first and second image, wherein the blending [further] includes: 

dividing the [second] third image into a first portion and a second portion, based on the 
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first positiori; and 

coupositing the first portion of the [second] third image on the first image at the first 
position to produce a composite image; the compositing causing the first portion to mask out a 
part of the first image[; 

blending the third image with the composite image to form the panoramic image]. 

27. (Amended) The article of claim 27 wherein blending the third image with the 
[composite image] first and second image further includes: 

dividing the [third] second image into a third portion and a second portion, based on [the 
second] a relative position of the second segment of the view relative to the first segment of the 
view ; 

dividing the third portion into a fifth portion and a sixth portion, based on the [third] 
second relative position; and 

compositing the fifth portion of the third image on the composite image based on the 
second relative position to form the panoramic image, the compositing of the fifth portion 
causing the fifth portion to mask out a part of the composite image. 

Claims 28 and 29 have been added. 

28. The method of claim 1 flirther comprising: 
prior to blending the set of images: 

determining which of the images is a central one and which are images; and 
using the central image as an initial reference image in correcting perspective 
distortion in peripheral images. 

29. The method of claim 28 further comprising: 

determining what pair- wise overlap areas exist between the central image and 
each of the peripheral images; and 

selecting as the first peripheral image to have perspective distortion corrected a 
peripheral image having a maximum pair-wise overlap area with the central image relative to the 
other peripheral images. 
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